FreezeOut: Accelerate Training by Progressively Freezing Layers
نویسندگان
چکیده
The early layers of a deep neural net have the fewest parameters, but take up the most computation. In this extended abstract, we propose to only train the hidden layers for a set portion of the training run, freezing them out one-by-one and excluding them from the backward pass. We empirically demonstrate that FreezeOut yields savings of up to 20% wall-clock time during training with 3% loss in accuracy for DenseNets on CIFAR.
منابع مشابه
Mahabad Spring Freezing Forecasting, Using Synoptic Study
The quality of pressure distribution over the atmosphere of the sea level and geopotential altitude at different atmosphere layers affect on many climate and meteorology phenomena of the earth surface. One of the important meteorology phenomenon is the occurrence of frost phenomenon especially late spring frost. In this surrey, occurring late spring frost days and minimum temperature during the...
متن کاملSpecifics of freezing of Lennard-Jones fluid confined to molecularly thin layers
Freezing of a Lennard-Jones fluid between solid surfaces was studied using grand canonical Monte Carlo and molecular dynamics simulations. We explored the formation of frozen phases of hexagonal and orthorhombic symmetry in mono-, bi-, and tri-layer structures. The freezing transition, the type of lattice, and translational and orientational ordering were identified on the basis of orientationa...
متن کاملChemical Freezeout in Heavy Ion Collisions
Derek Teaney Department of Physics, Brookhaven National Laboratory, Upton, NY 11973-5000 (Dated: February 8, 2008) Abstract We construct a hadronic equation of state consistent with chemical freezeout and discuss how such an equation of state modifies the radial and elliptic flow in a hydrodynamic + hadronic cascade model of relativistic heavy ion collisions at the SPS. Incorporating chemical f...
متن کاملMultiscale Hierarchical Convolutional Networks
Deep neural network algorithms are difficult to analyze because they lack structure allowing to understand the properties of underlying transforms and invariants. Multiscale hierarchical convolutional networks are structured deep convolutional networks where layers are indexed by progressively higher dimensional attributes, which are learned from training data. Each new layer is computed with m...
متن کاملProgressively Diffused Networks for Semantic Image Segmentation
This paper introduces Progressively Diffused Networks (PDNs) for unifying multi-scale context modeling with deep feature learning, by taking semantic image segmentation as an exemplar application. Prior neural networks such as ResNet [11] tend to enhance representational power by increasing the depth of architectures and driving the training objective across layers. However, we argue that spati...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- CoRR
دوره abs/1706.04983 شماره
صفحات -
تاریخ انتشار 2017